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13 April 1977 


MEMORANDUM FOR: Deputy Director for Administration 
Deputy Director for Intelligence 
Deputy Director for Operations 
Deputy Director for Science §& Technology 
Executive Secretary 


THROUGH : Comptroller 


SUBJECT :. Resources for Program Evaluation in Fiscal Year 1977 
~- OMB Bulletin 77-8  .- 


1. A report showing and commenting on the CIA resources devoted 
to Program Evaluation is due in the Office of Management and Budget on 
_25 April, It is in our interest that the CIA submission show as 
thoroughly and carefully as possible Agency efforts to systematically 
analyze its programs to ascertain the extent that they "have achieved or 
are achieving their objectives, or are producing other Significant 
effects .¢4."" 


2. A copy of the actual OMB Bulletin is attached; it requires 
some interpretation. Basically,:the data desired’ is that showing the 
extent that CIA attempts to determine "what has actually occurred as the 
result of past program actioris,'’ plus our efforts to develop evaluation 
‘methodologies and ensure that these are incorporated into existing and 
future programs. The efforts identified should be intended to assist 
management. 7 


3. For the purposes of this OMB tasking, program evaluation 
does not include: 


~~ evaluation activities which are an intrinsic part of pro- 
gram operations or management; 


-- analyses of the relative cost/benefit of alternative ~ 
approaches to programs that do not yet exist; 


-- the development and operation of management information 
systems; . 


-- the collection of routine data or the preparation of general 
statistics or basic research. 
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4. Under such criteria, we would assume that, for example, 
approaches to the development of a new technical collection system would 
not be included, but that the evaluation of performance of existing 
systems would be. Specia efforts to monitor the effectiveness of 
particular products would be Included when these were expected to be 
used as a basis for new policy and management decisions. Formal studies 
of the DDO Evaluation and Program Design Staff and others of collection 
projects would be included. The efforts of much of the DDI Center for 
Policy Support is aimed at determining the utility of finished intelli- 
gence. Some of the work of the DDA Center for the Study of Intelligence 
might be included. A portion of the efforts of the Inspections Staff 
and the Audit Staff deal with program evaluation. The NIOs engage in 
program evaluation when they participate in '"FOCUS-type" studies.or 
consider the performance of collectors relative to Key Intelligence 
Questions. 


5. The attachment to the OMB Bulletin shows the prescribed format 
for your input. Please give us a name of a contact person; we will not 
pass that name to OMB. We propose to provide OMB with data for each of 
the four Directorates and for the Office of the Director. One staff-year 
should be counted: for each full-time person who devotes more than fifty 
percent of his/her time to program evaluation activities or to monitoring 
such activities by contractors; part-time employees or employees working 
on evaluations less than fifty percent of their time are not to be 
included. _A brief narrative description of program evaluation activities 
and of their expected impact is to be included (two or three paragraphs 
per Directorate). Significant increases or decreases in funding levels 
for evaluation activities from FY 1977 to FY 1978 are to be noted and 


major "policy or strategy changes" mentioned. 


6. <A two-three page Agency summary narrative will be prepared by 
this office and we welcome your suggestions. The data you provide 
should include activities of your components in support of interagency 
evaluation efforts (e.g., by the IC Staff) or of evaluation activities 
of other agencies. The Agency overview will note that CIA regularly 
participates in evaluations of Programs under the control of other 
Departments and Agencies. 


7. Your data and narrative are needed by no later than COB 
21 April. We apologize for the tight deadline, but the OMB tasking only 
arrived in the Comptroller's Office on 12 April, and the OMB deadline is 


“Monday, 25 April, reais of my Staff (Red 1829, Rm 3E58), is. 
coordinating this effort and he remains ready to talk further with your 


people regarding the nature and extent of the task. 
25X1A 


Assistant Comptroller 
Requirements § Evaluation 


Attachment: 
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BULLETIN NO. 77-8 a April 4, 1977 


LO THE HEADS OF EXECUTIVE DEPARTMENTS, AND ESTABLISHMENTS 


SUBJECT: Resources for Program Evaluation in Fiscal Year 
1977 : 


1. Purpose. This Bulletin provides instructions for the 
Submission of information concerning the resources for 
program evaluation provided for in fiscal year 1977, 

2. Background. The Office of Management and Budget has 
determined the need to collect and analyze information on 
the level of Federal expenditures and Staffing devoted to 
program evaluation activities. The information requested by 
this Bulletin is vital to an understanding of the magnitude 
of evaluation resources and the wayS in which: these 
resources contribute to the achievement of national goals, 
agency missions, and program objectives. Moreover, 
increasing concern about the adequacy of data on program 
performance makes it necessary to assure that accurate 
information can he provided to the Congress and the public 
on the actual level of program evaluation expenditures. 


3. Coverage. These instructions apply to each executive 
department and independent agency with more than 800 
authorized full-time permanent employees. ; 


4. Definition. For purposes of this Bulletin, program 
evaluation means the formal assessment, through objective 
measurements and systematic analyses, of the manner. and 
extent to which Federal programs (or their components) have 
achieved or are achieving their objectives, or are producing 
other significant effects of either a desirable or 
undesirable character, in order to assist future policy and 
Management: decisions. 


a. The following will be considered as Federal program 
evaluation activities for the purpose of this Bulletin: 
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(1) . Formal studies, surveys, and data analyses for 
the purpose of determining the effectiveness, efficiency, or 
impact of a national or regional program. 


(2) Systematic assessment of demonstration prograns 

Or projects which are expected to have major implication for 

programs of national or regional scope; except that 

‘ evaluation. activities which are an intrinsic part of the 
program operations or management should not be reported. 


(3) Formal studies, surveys, and data analyses .for 
the purpose of assesSing program management processes, 
procedures, intervention techniques, workload distribution, 
Or BEOCMePEV DEY: 


(4) Assessment ‘and dewevonnene of program designs 
to assure that programs, once operational, can be 
successfully evaluated. 


{5) Design, development, and field sceakend of new 
program evaluation methodologies. 


(6) Syn nthesis -and further analysis of results > 
obtained by several previous program evaluation efforts. 


(7) Collection of initial data to help in 
evaluation design and provide a baseline for subsequent 
evaluations. 

b. The following will .not be considered as program 
evaluation activities for the p PEYDOsES of this BudTeeins 


(1) Design, development, and operation of Seneene? 
data systems or management information SYSLeMs'. 


(2) Continuing collection of - routine data and 
general purpose statistics. 

(3) Analysis of existing or proposed policies where 
no programs yet exist (even thcugh authorized) for purposes 
of appraising the likely costs and Perecte aoe feasible 
alternatives. Although such analyses ar ofte called 
"evaluations," they are prospective in character; whereas 
program evaluation is retrosvective, aimed at determining 

what. has actually occurred as the result of past program 
actions. 


(4) Basic research and studies intended to increase 
or foster general knowledge cevelopment, but which are not 
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pecte Specifically and primarily in policy and 


“Ianagement decisions. 


(5) Routine, day-to-day monitoring of program 
operations which is an intrinsic part of program 
administration. 


ec. It is recognized that the present extensive 
variation among Federal agencies and programs will require 
case-by-case judgments to determine the applicability of the 
above criteria to agency "evaluation" activities. 


9. Reporting Requirements. Departments and agencies should 


submit a report in accordance with the attached instructions 


and exhibit to OMB, attention: Evaluation and Program 
Implementation Division by April 25, 1977. ‘This is a one- 


time report. 


6. Rescission. This Bulletin is rescinded Eollowing 


receipt of the April 25, 1977 report. 


7. Inguiries. Inquiries and requests for assistance should 
be directed to the Evaluation and Program Implementation 
Division, Otfice of Management and Budget. (395-4726 or 


395-7642, ) 
i t 
ph — 


Bert Lance’ 
Director 


Attachment 
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INSTRUCTIONS 


An original and one copy of the report on program evaluation 
resources will be prepared on letter size sheets (8x10-1/2) 
and submitted in the format of the attached exhibit, as 
prescribed below. . ; 


Date ~ Enter the date of preparation 


Contact - Enter the name and telephone number of an 
individual .who can respond to questions relative to 
the information provided. a 


Part I. RESOURCES SCHEDULE: 


Department/Agency/Bureau ~ Enter the reporting department or 
independent agency. In the case of cabinet departments, 
enter each bureau or comparable. organizational unit for 
which separate evaluation resources (funds and personnel) 
can be identified, a consolidated line entry for any other 
Organizational units (if applicable), and a line entry for 
the total department or agency. At least one space should 
be left between each line entry. ‘ : 


Staff years ~ Enter the estimated staff-years for full-time 
permanent personnel who devote more than fifty percent of 
their time to the program evaluation activities outlined in 
paragraph 4a of this Bulletin, This includes full-time 
government personnel engaged in monitoring or directing such 
activities by contractors or grantees. . 


Salaries - Enter the estimated obligations for personnel 
compensation related to the stari-year entry. 


Personnel Benefits -~ Enter the estimated obligations for 
bersonnel benefits associated with the salary figure. 


Contracts/Grants ~ Enter the estimated obligations to be 
incurred for evaluation contracts or grants, or, in the case 


Of multi~purpose agreements, the obligations for those: 


portions of the contracts or grantS that are directly 
related. to the program evaluation activities described in 
paragraph 4a of this Bulletin. 


Other Expenses ~ Enter the total obligations estimated for 
otner administrative expenses directly related to the 
ecformance, commissioning, or Oversight of the activities 
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described in “paragraph 4a of this Bulletin. "Other 

expenses" would include obiigations for travel and 


transportation of persons, transportation of things, rent, 
communications and utilities, printing and reproduction, 
other services (such as ADP services), supplies and - 
materials, and equipment. Significant obligations for any 
of these "other expenses" should be briefly identified in 
the narrative. 


Total - Enter the total estimated FY 1977 obligations for 
Salaries, personnel benefits, contracts/grants, and other 
expenses. Note: All obligations are to be entered in 
thousands of dollars. 


Part II. NARRATIVE STATEMENT 


ee 


A prief explanatory statement should describe in general. 
terms the program evaluation activities being undertaken and 
the objectives toward which they are directed. It should 
highlight significant _ increases or decreases in funding 


levels. £rom the _current... fiscal year, to FY. A978 and major 
policy” or ‘Strategy _.changes-..expected.._to... occur. “Ehphas ts 


_snould™“be™™ placed on the expected use of evaluation 


information in the agency resource allocation and other 
decision processes. In most instances, the narrative for 
individual bureaus should be restricted to two or three 
paragrapns and the departmental or agency summary should not 
exceed two pages. : 
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EXHIBIT 2 
Bulletin No. 77-8 


REPORT ON FY 1977 PROGRAM EVALUATION RESOURCES 
(Dollars in Thousands) 


Date: 


Contact: Berea pe ose pd te ( 


Part I. RESOURCE SCHEDULE 


: Personnel’ Contracts Other 
Staff Years. Salaries Benefits Grants Expenses TOTAL 


es? ue, ot. as 2 


JL eote aN een, 


Department/Agency/Bureau 


EP Sr ement/Age meee 


Department of Government 
(Report breakdown of major 
bureau or other organiza- 
tional unit for cabinet 
departments only) 


PART II, NARRATIVE STATEMENT | ( 
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Existing ODP Program Evaluation Methods 


Management by Objectives Program 
Period - Bi-Monthly and Annually 


Scope - Projects of Agency, Directorate of Office 
importance which have Action Plan 


"Board of Directors" 
Period - Monthly and on Demand 


Scope - Anthing of interest to Director, initially 
follows MBO 


Program Call and Revision to Program Call 
Period - Annually and Cyclic 

Scope - Only that portion devoted to evaluation 
Project Continuance Review 

Period - Monthly (all projects once a year) 


Scope - All prodution systems for which ODP is 
responsible 


Project Managemelin) 

Period ~- According to system development schedule 

Scope - Review and evaluation of past activity and 
future plans, according to project schedule. Examples 
are feasibility study evaluation, preliminary design 
review, critical design review and acceptance tests, et. 
Hardware/Software Procurement Concurrence 

Various Ad Hoc Evaluation Groups 

Steering Committees 

GIMS Project Reviews 


Configuration Management 


Project Team Evaluation and reviews 
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Data Systems Which Aid in Program Evaluation 

- Project Activity Report 
Period - Monthly 
Scope - All projects which use ODP resources 

- Applications Tracking System 
Period - On-line, Weekly and Monthly Reports 
Scope - All projects which use Applications resources, 
and/or for which Applications is responsible, tracks 
progress. 

- SMF Accounting 


Period - Daily, Monthly, Annually 


Scope - Support all machine resource accounting 
systems 
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